System and method for a voice-controllable apparatus

ABSTRACT

In accordance with an embodiment, an apparatus includes a millimeter wave radar sensor system configured to detect a location of a body of a person, where the detected location of the body of the person defines a direction of the person relative to the apparatus; and a microphone system configured to generate at least one audio beam as a function at least of the direction.

This application claims the benefit of U.S. Provisional Application No.62/635,150, filed on Feb. 26, 2018, which application is herebyincorporated herein by reference in its entirety.

TECHNICAL FIELD

The present invention relates generally to audio systems, and, inparticular embodiments, to a system and method for a voice-controllableapparatus.

BACKGROUND

Voice controlled electronic systems are becoming increasingly morecommon in smart-phone applications and Internet connected devices.During operation, such systems listen to voice commands issued by a userand perform actions in response to such commands. For example, a voiceactivated system may play music, provide a verbal weather forecastadjust a thermostat setting or lighting in response to the user's vocalrequest. Such systems may be deployed to provide control of electronicequipment a smart home.

Ensuring accurate detection of vocal commands designed to trigger aspecific response can be challenging given the extremely varied spatialconfigurations of their surroundings. For example, noisy environments,such as a large crowded room with many people speaking at once may posedifficulties in detecting a user's verbal command.

SUMMARY

In accordance with an embodiment, an apparatus includes a millimeterwave radar sensor system configured to detect a location of a body of aperson, where the detected location of the body of the person defines adirection of the person relative to the apparatus; and a microphonesystem configured to generate at least one audio beam as a function atleast of the direction.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention, and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings, in which:

FIG. 1 is an illustration of an apparatus according to an embodimentplaced within a given facility;

FIG. 2 is an illustration of the apparatus of FIG. 1;

FIG. 3 illustrates a millimeter wave radar sensor subsystem of anapparatus according to an embodiment;

FIGS. 4 to 8 illustrate various configurations of the millimeter wavesensor system of an apparatus according to an embodiment;

FIG. 9A illustrates an audio beam generated in the context of anembodiment so as to capture audio input, and FIG. 9B illustrates abeamforming antenna microphone according to an embodiment;

FIG. 10 illustrates signals which are transmitted and received by amillimeter wave sensor system of the apparatus according to anembodiment;

FIGS. 11 to 13 illustrate processing steps used in a method according toan embodiment; and

FIG. 14 illustrates a block diagram of a processing system that can beused to implement embodiment processing systems and methods.

Corresponding numerals and symbols in different figures generally referto corresponding parts unless otherwise indicated. The figures are drawnto clearly illustrate the relevant aspects of the preferred embodimentsand are not necessarily drawn to scale. To more clearly illustratecertain embodiments, a letter indicating variations of the samestructure, material, or process step may follow a figure number.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

FIG. 1 illustrates an apparatus 10 according to an embodiment, placed ina facility 20.

The facility 20 may for example be a room, for instance in a home. Forinstance, the facility 20 or its vicinity, e.g. the home in question,includes pieces of equipment 15 adapted to be remotely controlled, forinstance by the apparatus 10 or another piece of equipment operativelycoupled thereto. To this end, for instance, the apparatus 10 and thepieces of equipment 15 are connected to a network 17, such as a homenetwork.

The facility 20 further includes one or more access point defined by oneor more doors 60.

The apparatus 10 is configured to receive and detect commands issued bypersons 50 located in the facility 20, for instance with a view ofremotely controlling the pieces of equipment 15, e.g. to turn them on oroff, vary one or more parameter of their operations, etc. In the contextof an embodiment, the ensemble of possible commands to which theapparatus 10 is configured to react includes vocal commands, i.e. spokencommands. Optionally, this ensemble of commands includes gestures, suchas hand gestures, face gestures, arms gestures and so on.

In reference to FIGS. 1 and 2, the apparatus 10 includes a millimeterwave radar sensor system 30, or radar system 30, and a microphone system40, such as a microphone array of plural microphones no. The apparatus10 further includes a processing module 70. The systems 30, 40 and theprocessing module 70 are operatively coupled together.

The radar system 30 is configured to detect a person 50 in the vicinityof the apparatus, this detection being configured to provide at least adirection of the person relative to the apparatus 10. For instance, thisdirection may be defined by an azimuth and an elevation relative to theapparatus 10.

In the context of an embodiment, the radar system 30 is advantageouslyconfigured to detect the body of a person 50. In other words, the radarsystem 30 focuses on the body of the person 50.

In addition or alternatively, the radar system 30 is configured todetect a person regardless of whether the person is performing apredetermined gesture for triggering a predetermined response from theapparatus. In other words, the detection does not depend on whether theperson is performing a specific gesture which is to be interpreted as acommand by the apparatus, e.g. with a view to command a piece ofequipment 15. The specific gesture in question is for instance one ofmaking a specific movement with a body part such as a hand, moving thelips, looking at a specific object or direction, and so on. In thissense, this detection configuration of the apparatus can be seen aspassive, i.e. does not rely on intentional actions carried out by thepersons to be detected. In various embodiments, this passive detectionis accomplished by performing a series of Macro-Doppler and/orMicro-Doppler radar measurements configured to determine whether aparticular object is a human being based on detected motion and detectedhuman vital signs.

It should be noted that this detection scheme may be a mode of detectionof the apparatus among a plurality of possible different modes, and thatthe apparatus may be configured for performing a detection based onanother mode, either simultaneously in parallel of the detection modementioned above, or at a different time. In particular, the radar system30 may also be configured to detect that a specific gesture forming acommand destined to the apparatus is performed by a detected person. Inother words, the detection mode is adapted not to prevent the detectionof commands issued by persons.

Here, by “detection”, it is meant that the radar system 30 is configuredto output sensor data which are representative of the detection of theperson in the vicinity of the apparatus 10. These data may be raw sensordata, for instance when the processing of the data is made by a piece ofequipment external to the radar system 30 as in FIG. 2 such as theprocessing module 70 included in the apparatus or even by a processingequipment external to the apparatus. Alternatively, these data may beprocessed data which characterize at least the direction of the personrelative to the apparatus, such as in terms of elevation and azimuth,e.g. when the data are processed by the radar system 30 itself.

Advantageously, the radar system 30 is configured to simultaneouslydetect a plurality of persons, optionally through detecting the locationof their body and regardless of whether one or more person is performinga predetermined gesture configured to trigger a predetermined responsefrom the apparatus. Each detected person defines a direction relative tothe apparatus which is indicative of the relative direction between theapparatus and the person, e.g. in terms of azimuth and angle.

Advantageously, in the context of an embodiment, the radar system 30 isalso configured to detect the distance of the persons 50 relative to theapparatus 10. In other words, in addition to providing directionalinformation between the person and the apparatus 10, it is also adaptedto provide distance information between the person and the apparatus 10.

The radar system 30 exhibits a coverage area around the apparatus 10within which it is adapted to detect a person. The coverage area 80 isdelimited radially by a maximum detection range R of the radar system30. This range R may correspond to the physical limitation of the radarsystem in terms of coverage. This coverage area 80 may be circular, butmay not be. For instance, the maximum detection range R is then definedas the smallest of all the maximum detection ranges the coverage areahas, i.e. its minimal radius. Another definition may be adopted, such asthe maximum value of the maximum detection ranges, i.e. the maximalradius of the coverage area 80.

This radius is for instance considered based on a projection of thecoverage area (which is for instance a volume), for instance on thefloor of the facility 20 or in a plane parallel thereto.

In FIG. 1, the coverage area 80 is depicted as having a 360° angularcoverage around the apparatus. In other embodiments, the angularcoverage is strictly inferior to 360°. In embodiments, the angularcoverage is 180°. In embodiments, the angular coverage may be strictlyinferior to 180°.

The operating detection frequency (or frequencies) of the radar system30 is for instance above 20 GHz. For instance, an operating detectionfrequency of the radar system 30 is 24 GHz. Another possible frequencyis 60 GHz. Other frequencies may be used.

The radar system 3 o includes a plurality of millimeter wave radarsensor subsystems 90, or subsystems 90. Each of them is configured tocover an angular sector of the angular coverage of the radar system 30,and they jointly cover the entirely angular coverage of the radar system30. Their respective angular sectors may overlap or not.

Each subsystem 90 is in practice configured, e.g. at least spatiallyarranged, to cover the corresponding sector.

In at least one embodiment, advantageously, the millimeter wave radarsensor system is adapted to vary the angular coverage dynamically. Inother words, regions of space which are covered by the radar system 30at a given time may not be covered at a different time, all things beingequal (in particular the arrangement of the apparatus 10 in space).

To accomplish this, in a configuration, the radar system 30 is adaptedto selectively activate and deactivate each millimeter wave radar sensorsubsystem.

In another configuration, the apparatus 10 is adapted to selectivelyinclude or not include sensor output data respectively generated by eachmillimeter wave radar sensor subsystem during a processing of sensoroutput data generated by the millimeter wave radar sensor system.

In a specific embodiment, the apparatus 10 may operate in a standbymode, in which only part of the angular sectors are active. Forinstance, at least one angular sector which is kept active contains anaccess point of the facility, such as the door 60. Therefore, theapparatus may keep tracking of meaningful events (somebody entering theroom) in a reduced power mode.

In practice, the radar system 30 may turn on or off each angular sectorin these embodiments.

In a given embodiment, at least one angular sector may not be adjacentto another angular sector. In other words, the angular coverage of theradar system may not form a connected space.

In reference to FIG. 3, each subsystem 90 includes an integrated circuit92, one or more transmit antennas 94, one or more receive antennas 96,and a substrate 98. In some embodiments, the one or more transmitantennas 94 are arranged in a transmit antenna array, and/or the one ormore receive antennas 96 are arranged in a receive antenna array.

The integrated circuit 92 and the antennas 94, 96 are arranged on thesubstrate 98, whose spatial orientation is chosen (as a function of theelements of subsystem that is) so that the subsystem 90 covers thechosen angular sector.

In some embodiments, at least two subsystems 90 share an integratedcircuit chip 92, a substrate 98, or both. In some embodiments, the radarsystem 30 includes a single substrate for all the subsystems 90.

The integrated circuit chips 92 are for instance radio frequencyintegrated circuit (RFIC) chips.

It should be noted that the radar subsystems 90 may be implemented as aplurality of antenna elements formed on one or more substrate withintegrated circuit chips coupled to the planar antenna elements. Inother embodiments, the radar subsystems may each include antennaelements and RF circuitry in a single package that is subsequentlyattached to the planar surface of the substrate 210.

Moreover, various configurations are possible in terms of number oftransmit and receive antennas for each subsystem 90 as shown in FIGS. 4to 8. On these figures, the angular sectors are referred to by theexpression “zones” A to D.

In FIGS. 4 to 6, each subsystem has its own substrate and integratedcircuit chip, and has one receive antenna and one transmit antenna. FIG.4 illustrates millimeter wave sensor system 400 according to anembodiment. As shown, each of zones A, B, C and D include an integratedcircuit chip 404 that is configured to produce a respective antenna beam402 via at least one antenna (not shown). In some embodiments,millimeter wave sensor system 400 includes LEDs 406 that are configuredto be illuminated when the presence of a person is detected by thesystem. For example, the LEDs in zone A may become illuminated in when aperson is detected in zone A, and may be turned-off when a person is notdetected in zone A.

FIG. 5 illustrates millimeter wave sensor system 500 according to afurther embodiment. Millimeter wave sensor system 500 is similar tomillimeter wave sensor system 400 shown in FIG. 4 with the addition of areceive antenna 502 and a transmit antenna 504 for each integratedcircuit 404 in each zone A, B, C and D. Antennas 502 and 504 are of atype known as Vivaldi.

In FIGS. 6 to 8, the antennas are of a type known as Yagi-Uda and areconfigured to radiate in an end-fire direction parallel to the plane ofthe antenna. FIG. 6 illustrates millimeter wave sensor system 600 thatincludes a receive antenna 602 of the Yagi-Uda type and a transmitantenna 604 of the Yagi-Uda type corresponding to each integratedcircuit 404 in zones A, B, C and D. Each antenna 602 and 604 has acorresponding antenna beam 402.

FIG. 7 illustrates millimeter wave sensor system 700 that includes tworeceive antenna 602 of the Yagi-Uda type and one transmit antenna 604 ofthe Yagi-Uda type for each zone A, B, C and D. As shown, the antennas602 and 604 of zones A and B share a single integrated circuit chip 706,and the antennas 602 and 604 of zones C and D share a single integratedcircuit chip 706.

FIG. 8 illustrates millimeter wave sensor system 800 that includes onereceive antenna 602 of the Yagi-Uda type and one transmit antenna 604 ofthe Yagi-Uda type for each zone A, B, C and D. As shown, the antennas602 and 604 of zones A and B share a single integrated circuit chip 706,and the antennas 602 and 604 of zones C and D share a single integratedcircuit chip 706.

It should be appreciated that the millimeter wave sensor systemsdepicted in FIGS. 4-8 are just a few examples of many possibleembodiment millimeter wave sensor systems. In alternative embodiments,the millimeter wave sensor system may have greater or fewer than foursectors, may have different numbers of transmit and receive antennas,and may have other types of transmit and receive antennas beside Vivaldiand Yagi-Uda antennas, such as patch antennas.

Advantageously, the radar system includes a Frequency-ModulatedContinuous Wave radar sensor, or FMCW radar sensor. A given subsystemmay form such a sensor. For instance, in some embodiments, eachsubsystem is FMCW-based.

In practice, a subsystem may be FMCW-based at least given theconfiguration of its integrated circuit chip.

When such a FMCW sensor is present, advantageously, the apparatus isconfigured to implement both a Macro-Doppler Filtering operation tosensor data generated by the FMCW radar sensor, whereby movements of theperson are detected, and a Micro-Doppler Sensing operation to the sensordata generated by the FMCW radar sensor, whereby vital signs of theperson are identified. Here, FMCW is presented merely as a possiblemodulation scheme for the radar sensor allowing the aforementioneddetection scheme. Other modulation schemes may be employed such as apulsed radar or a code modulated scheme which are publically knownmodulation schemes for millimeter wave radar sensors.

In some embodiments, the apparatus includes at least one Doppler radarsensor or a radar system configured to perform Doppler measurements. Agiven subsystem may form such a sensor. For instance, in someembodiments, each subsystem defines such a Doppler sensor.

When a Doppler radar sensor is present, the apparatus 10 isadvantageously configured to implement a processing operation to sensordata generated by said Doppler radar sensor, whereby movements of thebody of the person are thus detected.

Still in reference to FIGS. 1 and 2 and in further reference to FIG. 9A,the microphone system 40 is configured to generate one or more audiobeam 100 (FIG. 9A) via each of which the microphone system is configuredto receive audio inputs. These audio inputs are adapted to include thevocal commands mentioned above, and in general correspond to any soundwhich may be produced in the vicinity of the apparatus 10.

In addition, the microphone system 40 is configured, for one or moreperson 50, to generate at least one audio beam associated to the person50 as a function at least of the direction of the person provided by theradar system 30 so as to enhance the detection and reception of audioinputs generated by the corresponding person by the apparatus 10. Here,“generate an audio beam” is intended to mean providing a receptiondirection for the microphone system along which the sensitivity of themicrophone system is increased.

In other words, as a function at least of the direction of the personsdetermined based on the radar system 30, the microphone system 40 steersone (or more) audio beam 100 in the corresponding direction so as toenhance the reception of the audio inputs generated by the person, andtherefore of the vocal commands this person may issue to the apparatus10.

For instance, typically, this takes the form of the direction of themain lobe of an audio beam being aligned with the direction of theperson as detected by the radar system 30. This steering of the audiobeams 100 is for instance done through a known process. For instance,this is achieved through processing the audio inputs of the microphonesno through forming a combination of these audio inputs chosen to includerespective coefficients for these inputs that are dynamically adjustedbased on the direction of the corresponding person as provided by theradar system 30. In other words, during the processing of the inputsprovided by the microphone system, a respective weight of the audioinputs provided by each microphone is dynamically adjusted so as tosteer the sensitivity of the microphone system in the detecteddirection.

Advantageously, this is the case for each person as detected by theradar system 30, i.e. at least one audio beam 100 (typically one forinstance) will be generated for each person as a function of thecorresponding direction so as to enhance the detection of the audioinputs generated by the corresponding person.

Enhancing the detection of the audio inputs may further includeincreasing the signal to noise ratio. To this end, for instance, a noisecancellation process of the audio inputs received via the audio beam 100may be performed based on the audio inputs received by the audio beamand audio inputs received by another beam not defined as a function ofthe direction of the person, and which will therefore pickup backgroundnoise relative to the considered person.

Advantageously, the audio beam is adjusted over time based on thechanges of the direction of the person relative to the apparatus asdetected by the radar system 30. In other words, the audio beam tracksthe person as the latter moves in space.

Advantageously, the microphone system 40 is configured to generate anaudio beam associated to a person also as a function of the distance ofthe person 50 relative to the apparatus as captured by the radar system30.

For instance, the beamwidth of the main lobe of the corresponding beamis adjusted by the microphone system 40 based on the distance of theperson 50 relative to the apparatus 10. This is for instance donethrough a known process. Additionally and independently of thebeamwidth, the sensitivity of the microphones may be adjusted accordingto the detected distance, i.e. the range of the audio beam may be variedaccordingly. In the context of the later described beamformingtechniques, this may be realized by determining the employed filtersalso based on the detected distance.

FIG. 9B illustrates an embodiment beamforming microphone system 900 thatcan be used, for example, to implement microphone system 40. As shown,beamforming microphone system 900 includes microphone array 902 coupledto a beamformer 904. In various embodiments, microphone array 902includes a plurality of microphones 901 ₁, 901 ₂ and 902 _(N). Whilemicrophone array 902 is shown having three microphones for simplicity ofillustration, it should be understood that microphone array 902 caninclude any number of microphone greater than two. Each microphone 901₁, 901 ₂ and 902 _(N) may be implemented using microphone structuresknown in the art, including, but not limited to a MEMS microphone, acondenser microphone, a condenser electret condenser microphone (ECM), adynamic microphone. Beamformer 904 receives the output of eachmicrophone 901 ₁, 901 ₂ and 902 _(N), performs beamforming processing onthe microphone outputs, and produces a directional audio signal BeamOutthat represents an audio signal corresponding to the generated audiobeam. Such beamforming processing may be performed, for example, usingappropriate analog and/or digital signal processing to producedirectional audio signal BeamOut. Beamformer 904 may be implementedusing analog signal processing circuitry and/or digital circuitry suchas a processor configured to execute digital signal processingalgorithms, digital signal processor (DSP), dedicated digital logic, orother circuitry known in the art.

It should be noted that the distance of the person relative to theapparatus may be taken into account in any detection mode of the radarsystem, whether the one mentioned above, or one in which the detectionof the person is carried out e.g. when a specific gesture has been madeby the person.

In an embodiment, the microphone system 40 will steer an audio beamtowards the detected person 50 only if the distance of the personrelative to the apparatus satisfies a predetermine criterion relative toa predetermined threshold distance TH_(D) chosen strictly inferior tothe maximum detection range R.

In an embodiment, the microphone system 40 is configured to generate theaudio beam for a person 50 as a function of at least the direction ofthe person relative to the apparatus only if the detected distance ofthe person 50 relative to the apparatus 10 is inferior or equal to thepredetermine threshold distance TH_(D).

In other words, a specific condition pertaining to whether the person isclose enough to the apparatus 10 in the sense of a threshold chosen todefine a portion of the coverage area strictly smaller than the coveragearea is scrutinized to determine whether or not an audio beam is steeredin the direction of the person.

Therefore, if a person is detected in the inner circle defined by thethreshold TH_(D), an audio beam will be steered in his/her direction. Ifhe or she is detected but is in the outer ring defined between thedistance R and the distance TH_(D), no audio beam is then steered inhis/her direction.

The threshold distance TH_(D) is for instance chosen greater or equalthan 10% of the distance R, 30 %, or even 50%. For instance, it iscomprised between 25% and 75% of the distance R.

In an embodiment, the microphone system 40 is configured to generate theaudio beam for a person 50 as a function of at least the direction ofthe person relative to the apparatus only if the detected distance ofthe person 50 relative to the apparatus 10 is greater than or equal to apredetermined threshold distance TH_(D).

In other words, a specific condition pertaining to whether the person isfar enough from the apparatus 10 in the sense of a threshold chosen todefine a portion of the coverage area strictly smaller than the coveragearea is scrutinized to determine whether or not an audio beam is steeredin the direction of the person.

Therefore, if a person is detected in the inner circle defined by thethreshold TH_(D), an audio beam will not be steered in his/herdirection. If he or she is detected in the outer ring defined betweenthe distance R and the distance TH_(D), then an audio beam is steered inhis/her direction.

In some embodiments, the two conditions above can be selectivelyactivated and deactivated.

In some embodiments, a same apparatus 10 may be configured to apply oneor the other condition depending on the chosen operating mode for theapparatus 10.

Although the same reference sign TH_(D) has been used for describingboth conditions, the respective distance values may of course bedifferent in one case and the other.

In an embodiment, the subsystems 90 each have a threshold distanceassociated thereto, and, for a person detected in the angular sector ofa given subsystem, the microphone system 40 will steer an audio beamtowards the person only if the distance of the person relative to theapparatus satisfies a predetermine criterion relative to the associatedthreshold distance.

For instance, it may steer the audio beam only if this distance isgreater than the threshold. Alternatively, it may steer the audio beamonly if the distance is inferior to the threshold distance.

In an embodiment in which the radar system 30 includes a first and asecond subsystems 90 each having a threshold distance TH_(D)′, TH_(D)″associated thereto,

-   -   for the first subsystem, the microphone system 40 is configured        to generate the audio beam for a person 50 detected in the        angular sector of the first subsystem only if the detected        distance of the person 50 relative to the apparatus 10 is        greater than or equal to the associated predetermined threshold        distance TH_(D)′, TH_(D)″; and    -   for the second subsystem, the microphone system 40 is configured        to generate the audio beam for a person 50 detected in the        angular sector of the second subsystem only if the detected        distance of the person 50 relative to the apparatus 10 is        smaller than or equal to the associated predetermined threshold        distance TH_(D)″, TH_(D)′.

In other words, the triggering effect on the audio beam steering by themicrophone system of the relationship between the distance of the personand the threshold distance may differ based on the subsystem.

In an embodiment, to each subsystem 90 may be associated a respectivethreshold distance. The respective threshold distances TH_(D)′,TH_(D)″of two subsystems may be different, as illustrated on FIG. 1.

In an embodiment, advantageously, one or more threshold distance of theapparatus 10, e.g. that of the radar system or the respective thresholddistances of the subsystems, may be defined to vary. For instance, thisvariation is implemented based on a usage history of the apparatus.

For instance, the usage history includes data indicative of distancesbetween the apparatus and persons at times at which the correspondingpersons issued commands to the apparatus, e.g. vocal commands.

In other words, these data represent distances between the apparatus andpersons at times at which these persons issued commands for theapparatus.

For instance, whether the usage history only includes these data forvocal commands, or only the data of the usage history that refer tovocal commands are taken into account.

Any operation or set of operations may be applied to the data todetermine the threshold distance(s), such as an averaging step, asampling step, a transformation step, etc.

For a given subsystem, only the data of the usage history referring tocommands issued from positions within the corresponding angular sectormay be taken into account.

In reference to FIG. 2, the microphone system 40 includes a plurality ofmicrophones no which are each adapted to form at least one steerableaudio beam 100 to receive audio inputs.

The audio beams may be mechanically steerable, electronically steerable,or both.

For instance, the microphones no are arranged, e.g. spatially, so as tocover at least the angular coverage of the radar system 30 around theapparatus 10.

Optionally, the microphone system 40 includes a control module 120configured to cause the steering of the audio beams of the microphonesno.

All or part of the functionalities of this control module 120 may beimplemented by the processing module 70.

In further reference to FIG. 2, the processing module 70 is configuredto handle the operations of the various elements of the apparatus 10 forthe normal operations thereof, in particular that of the radar system 30and of the microphone system 40.

In an advantageous embodiment, the processing module 70 is configured toprocess the sensor data generated by the radar system 30 or the audiodata collected by the microphone system 40, or both. As indicated above,this processing may be carried out directly by the radar system 30and/or by the microphone system 40.

In some embodiments, the processing module 70 may be external to theapparatus 10 and may be operatively coupled to the apparatus forcarrying out operations remotely, such as processing of data.

In general, various configurations in terms of distribution of theprocessing module 70 between the radar system 30 and the microphonesystem 40 are possible, ranging from a fully distributed configurationwherein all the functionalities pertaining to these systems 30, 40 areimplemented directly by them (and therefore each system may include onemore processing element) to a centralized configuration wherein all thefunctionalities that pertain for instance to processing data areimplemented by the processing module 70 which is then external to thesesystems 30, 40 (but may or may not be external to the apparatus 10).

In general, the processing module 70 may include a microprocessor or amicrocontroller.

In an embodiment, and as suggested above, the processing module 70 is inparticular configured to implement the processing of the sensor data ofthe radar system 30 and of the audio data generated by the microphonesystem 40.

As discussed above, the output of the processing of the data of theradar sensor is adapted to include direction information thatcharacterizes a direction of a person detected by the radar system 30relative to the apparatus 10.

Advantageously, it also includes a distance information indicative ofthe distance between this person and the apparatus 10.

At least the direction information, and advantageously also the distanceinformation, are then used in input by the microphone system 40 (or theprocessing module 70) to define the one or more audio beams generated bythe microphone array 40 as discussed above. The generation of the audiobeams per se may be implemented through any known process. In generatingthe audio beam, a microphone array is used to form a spatial filterwhich can extract a signal from a specific direction and reduce thecontamination of the signals from other directions, i.e. a source ofinterest may be selected while minimizing undesired interfering signalswhich may also be called beamforming. In various embodiments, themicrophone array may be implemented using microphone array 902 shown,and the various beamforming algorithms described below may beimplemented by beamformer 904 shown in FIG. 9B.

The used beamforming algorithms may for instance concentrate onenhancing the sum of the desired sources while treating all othersources as interfering sources. Therein beamforming may be accomplishedby filtering the microphone signals and combining the outputs to extract(by constructive combining) the desired signal and reject (bydestructive combining) interfering signals according to their spatiallocation, i.e. beamforming may separate sources with overlappingfrequency content that originate at different spatial locations.Generally the employed beamforming algorithms may include so calleddeterministic beamforming approaches as well as statistically optimumapproaches which may be discriminated by the technique used to estimatethe spatial filter weights. Both approaches can be used in the timedomain or frequency domain. In time domain beamforming, a finite impulseresponse (FIR) filter is applied to the microphone signal and the filteroutputs combined to form the beam, i.e. the beamformer output. Infrequency domain beamforming the microphone signal is e.g. separatedinto narrowband frequency bins using a short time Fourier transform(STFT) and the data in each bin is processed separately. Additionally,the aforementioned deterministic beamforming approaches may also becalled data-independent because their filters do not depend on themicrophone signals and are chosen to approximate a desired response.Conversely the mentioned statistically optimum beamforming approachesmay also be called data-dependent since their filters are designed basedon the statistics of the arriving data to optimize a predeterminedfunction that makes the beamformer optimum in a desired sense. For thedeterministic beamformer for instance one may wish to receive any signalcoming from a certain direction, in which case the desired response maybe unity at that direction. As another example, signals from anotherdirection may be treated as interference, in which case the desiredresponse at that direction is zero. One simple deterministic beamformeris publically known as delay and sum beamforming, where the signals atthe microphones are delayed and then summed in order to combine thesignals arriving from a certain direction of the desired sourcecoherently, expecting that the interference components arriving from offthe desired direction cancel to a certain extent by destructivecombining. Assuming that the broadband signal can be decomposed intonarrowband frequency bins as described above, the delay can beapproximated by phase shifts in each frequency band. A more generalbeamformer is publically known as filter-and-sum beamforming, where,before summation, each microphone signal is filtered with FIR filter oforder M (M being the number of microphones in the array). This approachmay be preferential of the simpler delay-and-sum algorithm in multipathenvironments, namely reverberant enclosures. For the statisticallyoptimum beamformers, which may also be used in some embodiments, aredesigned based on the statistical properties of the desired andinterference signals, wherein several criteria can be applied, such asmaximum signal to noise ratio (MSNR), minimum mean squared error (MMSE),minimum variance distortionless response (MCDR), and linear constrainminimum variance (LCMV). As a specific example of a MMSE estimate, themultichannel Wiener filter may be mentioned. As another specific system,which is related to an MVDR beamformer, the Generalized Side-lobeCanceller may further be mentioned. Any of the aforementioned and otherpublically known methods may of course be employed in embodiments of theinvention.

FIG. 10 illustrates signals which are transmitted and received by theradar system 30 according to an embodiment in operation, and which areprocessed to detect the persons 50 in the coverage area 80.

FIG. 10 illustrates a waveform diagram of an FMCW system. Signal 1022represents the frequency of the radar signal transmitted by atransmitter of the FMCW system, signal 1024 represents the frequency ofthe signal reflected by a detected target. The delay from thetransmission of the transmit signal to the receipt of the signalreflected by the target is Δt₂ at time t₁. Time delay Δt₁causes afrequency offset Δf₁ between the transmitted signal 1022 and thereceived signal 1024. Assuming that the target moves closer to the radarsensor at time t₂, the delay from the transmission of the transmitsignal to the receipt of the signal reflected by the target is Δt₂,which corresponds to a frequency offset Δf₂ between the transmittedsignal 1022 and the received signal 1024. Because the target is closerto the radar sensor at time t₂ than at time t₁, the corresponding timedelay Δt₁ and frequency offset Δf₂ for the target at time t₂ are lessthan the corresponding time delay Δt₂ and frequency offset Δf₁ for thetarget at time t₁.

In various embodiments, the transmitted signal is mixed with thereceived signal to create an intermediate frequency signal thatrepresents the difference in frequency between the transmitted signaland the received signal. As shown, the bandwidth B of the FMCW radarsystem is related to the difference between the maximum and minimumtransmitted signal.

FIG. 11 illustrates an example of an algorithm 1200 that can be appliedto the sensor data generated by the radar system 30 in embodiments ofthe present invention.

Advantageously, the apparatus 10 is configured to implement at least oneof a Macro-Doppler processing operation to the sensor data of the radarsystem 30 for detecting movements of a person and a Micro-Dopplerprocessing operation to these data for detecting vital signs of aperson.

At least one of these, for instance the Micro-Doppler processingoperation is performed to detect passive humans, i.e. humans that arenot moving in space deliberately, for instance humans who are sitting,standing still, lying e.g. for sleeping, etc.

The details of the operations of algorithm 1200 are as follows. In step1202 raw digital data is obtained by an analog to digital conversion ofthe sensor data. This step may be accomplished, for example using ananalog to digital converter known in the art. In step 1204, the obtainedsignal is conditioned using, for example, a known process. For instance,in once embodiment the obtained signal is filtered, DC components areremoved, and the IF data is cleared. In some embodiments, IF data iscleared by filtering to remove the transmit-receive self-interferenceand optionally pre-filtering the interference colored noise. In someembodiments, filtering includes removing data outliers that havesignificantly different values from other neighboring range-gatemeasurements. In a specific example, a Hampel filter is applied with asliding window at each range-gate to remove such outliers.Alternatively, other filtering for range preprocessing known in the artmay be used.

In step 1206, range-gates are selected, e.g. by selecting range-gateswhose mean is greater than the mean of all the other range gates in thefield-of-view as potential target range-gates, wherein in someembodiments a range FFT is taken of the conditioned radar data. In anembodiment, a windowed FFT having a length of a chirp may be calculatedalong each waveform. Each point of the range FFT may represent adistance between a millimeter-wave sensor and a detected object andcorresponds to a range gate. In some embodiments, a range FFT isperformed for radar data produced by each receive antenna in a receiveantenna array. In various embodiments, the range-gate selection alsodetermines the angle or azimuth of detected targets with respect to amillimeter-wave radar sensor as well as their range or distance to themillimeter-wave radar sensor.

In step 1208, range clustering is performed, for example, regroupingranges together, e.g. through a nearest neighbor clustering operation.In step 1210, Macro-Doppler detection is used to detect movements of aperson, e.g. through a Doppler Fast Fourier Transform and thresholdingperformed over selected range gates. In an embodiment, in thethreshold-based approach the short-time energy of the moving variance ofthe range-gate is examined. This variance energy may be empiricallycalculated in some embodiments. Range-gate measurements that fall belowthe threshold established by the short-time energy of the movingvariance of the range-gate are considered to be representative of staticobjects, while range-gate measurements that are above the threshold areconsidered to be representative of moving objects or environmentalchanges.

Vital signal identification is performed in step 1212, e.g. through aMicro-Doppler process using one or more low bandwidth filter to extractheart-beat signal and/or breathing signal from the selected range gatesif no movement is detected in the Macro Doppler detection. This allowsthe filtering out of static inanimate targets with high radar crosssection (RCS), as they would produce no vital signals/noise afterpassing through these filters, contrary to humans. In an embodiment, twofixed, calibrated low bandwidth filters are employed to extract aheart-beat signal and a breathing signal from the selected range gates.For example, a band-pass filter centered around 0.4 Hz with a bandwidthof 0.5 Hz can be used. Alternatively other center frequencies andbandwidths may be used. In some embodiments, vital signal identification1212 includes substep 1214 in which only monitoring is performed. Nestin substep 1218, the vital signal Micro-Doppler filters as describeabove are applied to signals monitored during step 1214. Lastly, insubstep 1218, the breathing signal is detected. Alternatively, vitalsignal detection may occur using different steps. In some embodiments,vital signal processing methods may be used that are disclosed in U.S.patent application Ser. No. 15/872,677, which has been incorporated byreference in its entirety.

The steps starting from the range clustering 1208 are then for instanceiterated over time to track changes of the direction of the personrelative to the apparatus.

Appropriate filtering, for instance through thresholding of the obtainedelevation or of an obtained heart beat/breathing cycle, may then beapplied to filter out living beings which are not humans, such as housepets and/or to distinguish between adults and children. For instance, ifthe estimated breathing cycle is not within a predetermined range thatcorresponds with a normal human respiration, for example, between about12 breaths per minute and about 35 breaths per minute, then it may bedetermined that the target is not human. As a further example, if theestimated heart rate is not within a predetermined range thatcorresponds with a normal heart rate, for example, between about 50beats per minute and about 200 beats per minute, then it may bedetermined that the target is not human.

In more detail, in some embodiments, during operation, themillimeter-wave radar system 40 first performs a coarse measurementusing Macro-Doppler techniques to determine the presence of moving andnon-moving objects. (In some embodiments, non-vital motion iscategorized using Macro-Doppler techniques.) Next, the millimeter-waveradar system 40 performs a series of more targeted measurements of thedetected objects using Micro-Doppler techniques to determine whetherthese detected objects exhibit a heart-rate and respiration within theexpected range of a human being.

In embodiments that utilize a frequency modulated continuous wave (FMCW)radar sensor, the location of each object in the facility 20 within arange-gate may be found by taking a range (fast Fourier transform) FFTof the baseband radar signal produced by a millimeter-wave radar sensor90 of the millimeter wave radar system 40, and the motion of the variousobjects may be determined, for example, by taking further FFTs todetermine each object's velocity using Doppler analysis techniques knownin the art. In embodiments in which a millimeter-wave radar sensor 90 ofthe millimeter wave radar system 40 includes a receive antenna array,further FFTs may also be used to determine the azimuth of each objectwith respect to the respective millimeter-wave radar sensor. Forexample, furniture may be identified in a range-gate as being a staticobject, a fan may be identified in a range-gate as being a movingobject, a static human may be identified in a range-gate as being astatic object and a moving human may be identified in a range-gate asbeing a moving object.

In some embodiments a two-dimensional FFT may be taken of a range FFTover slow-time to determine the velocity of each detected object.Alternatively, the velocity of each object may be determined by otherwaveform techniques including, but not limited to triangular chirp andstaggered pulse repetition time (PRT).

Next, Micro-Doppler techniques are used to detect small motions of eachabove mentioned object. These small detected motions are analyzed todetermine whether these motions are indicative of the heart rate andrespiration of a human being. Therein, the millimeter wave radar system40 makes a series of radar measurements that are more specificallydirected toward each object. For example, in embodiments in which amillimeter-wave radar sensor 90 of the millimeter wave radar system 40includes a transmit antenna array, these directed measurements areperformed by steering the radar beam produced by a millimeter-wave radarsensor 90 of the millimeter wave radar system 40 using phase-array radartechniques. In some embodiments, beamforming may utilize discreteprolate spheroidal sequence (DPSS) methods or other beamforming methodsknow in the art. Based on these more directed radar measurements, theprocessing module 70 determines whether each object experiences smallmotions consistent with human vital signs such as heart rate andrespiration. For example, the aforementioned furniture and fan maybeidentified as non-living objects for exhibiting no human-like vitalsigns, whereas the aforementioned moving human is recognized as a humanobject via a Macro-Doppler classifier, meaning that the motion of themoving human exhibits human-like motion. The aforementioned static humanis not recognized as a moving object but exhibits human-like vitalsigns.

In some embodiments, the results of the Macro-Doppler filtering areinput to a machine learning algorithm such as, but not limited to arandom forest algorithm, adaptive boosting (AdaBoost) algorithm and/or aneural network algorithm in order to identify the type of object beingdetected. Similarly, the vital signals determined in Micro-Dopplersensing stage 306 may also be input to the machine learning algorithm inaddition to the Macro-Doppler data to distinguish and identify objectssuch as moving human beings and other objects such as robots andanimals. For instance, the above steps are advantageously applied to thedata which are generated by FMCW sensors. Pulsed radar sensors, codemodulation, non-linear frequency modulation (NLFM), or Monte Carloforecasting of waves (MCFW) may also be employed.

FIG. 12 illustrates an example of a further algorithm 1300 that can beapplied to the sensor generated by radar system 30 in embodiments of thepresent invention. In some embodiments, the processing of the data mayinclude a reduced number of steps that include obtaining the raw digitaldata by analog to digital conversion of the radar sensor output (step1302), conditioning the resulting signal (step 1204), and applying aMacro-Doppler detection (step 1210) in order to detected a humanpresence (1220). Steps 1202, 1204, 1210 and 1220 are described abovewith respect to FIG. 11. Algorithm 1300 may be advantageously applied tothe sensor data provided by Doppler sensors.

In an embodiment, further processing steps of the data may beimplemented for one or more radar sensor according to method 1400 shownin FIG. 13. In step 1404, a plurality of persons 50 are detected by theradar sensor 30. Next, in step 1406, data is determined for eachdetected person 50, in a particular azimuth, elevation and range (e.g.distance). A Householder transformation is then applied to the azimuth,elevation and range in step 1408. Next, the results of the Householdertransformation for each sensor is aligned in step 1410 so as totranspose the obtained result in a referential made common for all thesubsystems in step 1408.

These operations may be implemented for the output data provided by agiven sensor regardless of whether several persons are detected, forinstance so as to provide the information based on which the microphonesystem 40 will steer the audio beams 100 in a chosen referential.

A method according to an embodiment will now be described in referenceto the Figures.

In a general configuration, a method includes:

-   -   using the radar system 30, detecting one (or more) person 50 so        as to obtain a direction of the person relative to the        apparatus,    -   using the microphone system 40, steering at least one audio beam        100 as a function at least of the direction so as to enhance the        detection by the microphone of audio inputs generated by the        detected person 50 and received by the microphone system via the        generated audio beam, and    -   receiving audio inputs generated by the person via the steered        audio beam.

Advantageously, when only the direction of the person(s) is used foraudio beam steering, the method includes:

-   -   using the radar system 30, detecting the location of the body of        one or more person 50 regardless of whether the person is        performing a predetermined gesture configured to trigger a        predetermined response from the apparatus, whereby the detected        location of the body of the person defines a direction of the        person relative to the apparatus,    -   using the microphone system 40 configured to generate one or        more audio beam 100 via each of which the microphone system is        configured to receive audio inputs, generating at least one        audio beam 100 as a function at least of said direction so as to        steer said at least one audio beam 100 in said direction and        enhance detection of audio inputs generated by the detected        person 50 and received by the microphone system via the        generated audio beam, and    -   receiving audio inputs generated by the person via the generated        audio beam.

Advantageously, when both the direction and distance of the person(s) isused for the operations of the microphone system 40, the methodincludes:

-   -   using the radar system 30, detecting a person 50, this detection        providing at least a direction of the person relative to the        apparatus and a distance of the person relative to the        apparatus,    -   using a microphone system configured to generate one or more        audio beam via each of which the microphone system is configured        to receive audio inputs, generating at least one audio beam as a        function at least of said direction and said distance of the        person relative to the apparatus so as to enhance detection of        audio inputs generated by the detected person and received by        the microphone system via the generated audio beam, and    -   receiving audio inputs generated by the person via the generated        audio beam.

The various modalities and functionalities of the radar system 30, themicrophone system 40 and processing module 70 discussed above may thenbe implemented whether separately or in combination according to anypossible technically compatible combination regardless of the specificmethod applied.

In particular, advantageously, the process implements the principle ofonly steering an audio beam in the direction of the detected person (andpossibly generating the beam also as a function of the distance betweenthe apparatus and the person) only the distance between the person andthe apparatus obeys a specific relationship relative to the or onethreshold distance of the apparatus, for instance pertaining to thesubsystem in the angular sector of which the person is detected. One ormore thresholding operation of the distance provided by the radar system30 may be performed to this end.

Advantageously, the processing of the sensor data generated by the radarsystem includes applying a gaussian kernel density estimation (KDE)method to take into account shadowing phenomena that may occur due tothe output of a plurality of radar sensors being used.

In some embodiments, the apparatus 10 may be configured to provide oneor more stimulus indicative at least of the fact that a person has beendetected by the device, and preferably also of the direction of theperson relative to the apparatus, e.g. of its azimuth, elevation orboth.

For instance, to this end, the apparatus 10 includes visual elementssuch as LEDs configured to be turned on or off so as to reflect this orthese facts. For instance, they are spread around a circumference of oneor more surface of the apparatus, the location of the active visualelements being indicative of the direction of the person who has beendetected.

The apparatus 10 may also or alternatively include one or more speakersconfigured to provide an audio stimulus in response to the detection ofthe person. This stimulus may take the form of words spoken for theattention of the detected person.

Referring now to FIG. 14, a block diagram of a processing system 1500 isprovided in accordance with an embodiment of the present invention. Theprocessing system 1500 depicts a general-purpose platform and thegeneral components and functionality that may be used to implementportions of the embodiment radar and audio system and/or an externalcomputer or processing device interfaced to the embodiment radar andaudio system. The processing system 1500 may include, for example, acentral processing unit (CPU) 1502, memory 1504, and a mass storagedevice 1506 connected to a bus 1508 configured to perform the processesdiscussed above. The processing system 1500 may further include, ifdesired or needed, a video adapter 1510 to provide connectivity to alocal display 912 and an input-output (I/O) Adapter 1514 to provide aninput/output interface for one or more input/output devices 1516, suchas a mouse, a keyboard, printer, tape drive, CD drive, or the like. Invarious embodiments, CPU 1502 executes instructions in an executableprogram stored, for example in a non-transitory computer readablestorage medium, such as a memory 1504 to perform the various functionsof embodiment systems.

The processing system 900 also includes a network interface 1518, whichmay be implemented using a network adaptor configured to be coupled to awired link, such as an Ethernet cable, USB interface, or the like,and/or a wireless/cellular link for communications with a network 1520.The network interface 1518 may also comprise a suitable receiver andtransmitter for wireless communications. It should be noted that theprocessing system 1500 may include other components. For example, theprocessing system 1500 may include power supplies, cables, amotherboard, removable storage media, cases, and the like. These othercomponents, although not shown, are considered part of the processingsystem 1500.

While this invention has been described with reference to illustrativeembodiments, this description is not intended to be construed in alimiting sense. Various modifications and combinations of theillustrative embodiments, as well as other embodiments of the invention,will be apparent to persons skilled in the art upon reference to thedescription. It is therefore intended that the appended claims encompassany such modifications or embodiments.

What is claimed is:
 1. An apparatus comprising: a millimeter wave radarsensor system configured to detect a location of a body of a personregardless of whether the person is performing a predetermined gestureforming a command for triggering a predetermined response from theapparatus, wherein the detected location of the body of the persondefines a direction of the person relative to the apparatus; and amicrophone system configured to generate one or more audio beam via eachof which the microphone system is configured to receive audio inputs,the microphone system being configured to generate at least one audiobeam as a function at least of said direction so as to steer said atleast one audio beam in said direction and enhance detection of audioinputs generated by the person and received by the microphone system viathe generated audio beam.
 2. The apparatus of claim 1, wherein: themillimeter wave radar sensor system is configured to have apredetermined angular coverage around the apparatus; and the millimeterwave radar sensor system is adapted to vary the angular coveragedynamically.
 3. The apparatus of claim 2, wherein for varying theangular coverage dynamically, the millimeter wave radar sensor system isconfigured to selectively activate and deactivate each millimeter waveradar sensor subsystem.
 4. The apparatus of claim 2, wherein for varyingthe angular coverage dynamically, the apparatus is configured toselectively include or not include sensor output data respectivelygenerated by each millimeter wave radar sensor subsystem during aprocessing of sensor output data generated by the millimeter wave radarsensor system.
 5. The apparatus of claim 1, wherein for detecting thelocation of the body of the person, the apparatus is configured toimplement at least one of a Macro-Doppler processing operation fordetecting movements of the person and a Micro-Doppler processingoperation for detecting vital signs of the person.
 6. The apparatus ofclaim 1, wherein the microphone system is configured to adjust over timethe configuration of the audio beam via which audio inputs generated bythe person are received as a function of changes of at least thedirection of the person relative to the apparatus detected by themillimeter wave radar sensor system.
 7. The apparatus of claim 1,wherein the millimeter wave radar sensor system is configured tosimultaneously detect respective locations of the bodies of a pluralityof persons regardless of whether one or more person is performing thepredetermined gesture forming the command for triggering thepredetermined response from the apparatus, wherein the detectedlocations of the bodies of the person define respective directions ofthe persons relative to the apparatus, and wherein the microphone systemis configured to generate, for each of said persons, an audio beam as afunction at least of the corresponding direction so as to enhancereception of audio inputs generated by the corresponding person andreceived by the microphone system via the generated audio beam.
 8. Theapparatus of claim 7, wherein the millimeter wave radar sensor system isconfigured to detect a distance of the person relative to the apparatus;and wherein the millimeter wave radar sensor system has a maximumdetection range, and wherein the microphone system is configured togenerate the audio beam for the person as a function of at least thedirection of the person relative to the apparatus only if the detecteddistance of the person relative to the apparatus satisfies apredetermined criterion relative to a predetermined threshold distancechosen strictly inferior to the maximum detection range.
 9. Theapparatus of claim 8, wherein each millimeter wave radar sensorsubsystem has a predetermined threshold distance associated thereto,and, for a person detected in an angular sector of a given millimeterwave radar sensor subsystem, the microphone system is configured togenerate the audio beam for the person as the function at least of thedirection of the person relative to the apparatus only if the distanceof the person relative to the apparatus satisfies a predeterminedcriterion relative to the threshold distance associated to thecorresponding millimeter wave radar sensor subsystem.
 10. The apparatusof claim 9, wherein the respective threshold distances of at least twomillimeter wave radar sensor subsystems are different.
 11. The apparatusof claim 9, wherein: for a first millimeter wave radar sensor subsystem,the microphone system is configured to generate the audio beam for aperson detected in an angular sector of the first millimeter wave radarsensor subsystem as a function of the direction of the person relativeto the apparatus only if the distance of the person relative to theapparatus is greater than or equal to the predetermined thresholddistance associated to the first millimeter wave radar sensor subsystem;and for a second millimeter wave radar sensor subsystem, the microphonesystem is configured to generate the audio beam for the person detectedin the angular sector of the first millimeter wave radar sensorsubsystem as the function of the direction of the person relative to theapparatus only if the distance of the person relative to the apparatusis inferior or equal to the predetermined threshold distance associatedto the second millimeter wave radar sensor subsystem.
 12. The apparatusof claim 8, wherein the apparatus is configured to vary the thresholddistance as a function of a usage history of the apparatus whichincludes data indicative of distances between the apparatus and personsat times at which the persons issued at least one command to theapparatus.
 13. The apparatus of claim 8, wherein the microphone systemis configured to generate the audio beam for the person only if thedetected distance of the person relative to the apparatus is greater orequal to the predetermined threshold distance.
 14. The apparatus ofclaim 8, wherein the microphone system is configured to generate theaudio beam for the person only if the detected distance of the personrelative to the apparatus is inferior or equal to the predeterminedthreshold distance.
 15. The apparatus of claim 7: wherein the millimeterwave radar sensor system is configured to detect a distance of theperson relative to the apparatus; and wherein the millimeter wave radarsensor system has a maximum detection range, and wherein the microphonesystem is configured to generate the audio beam for the person as afunction of at least the direction of the person relative to theapparatus only if the detected distance of the person relative to theapparatus is superior or equal to a predetermined threshold distancechosen strictly inferior to the maximum detection range.
 16. Theapparatus of claim 1, wherein the apparatus is configured to perform anoise cancellation operation to enhance the detection of the audioinputs generated by the person.
 17. The apparatus of claim 1, whereinthe millimeter wave radar sensor system is configured to detect a sizeof the person, and the apparatus is configured so that the audio beamfor the person is only generated by the microphone system if the size ofthe person is greater or equal than a predetermined threshold size. 18.An apparatus comprising: a millimeter wave radar sensor systemconfigured to detect a person, so as to obtain a direction of the personrelative to the apparatus and a distance of the person relative to theapparatus; and a microphone system configured to generate one or moreaudio beam via each of which the microphone system is configured toreceive audio inputs, the microphone system being configured to generateat least one audio beam as a function at least of said direction andsaid distance of the person relative to the apparatus so as to enhancedetection of audio inputs generated by the detected person and receivedby the microphone system via the generated audio beam.
 19. The apparatusof claim 18, wherein the millimeter wave radar sensor system has amaximum detection range, and wherein the microphone system is configuredto generate the audio beam for the person as the function of at leastthe direction and the distance of the person relative to the apparatusonly if the detected distance of the person relative to the apparatusverifies a predetermined criterion relative to a predetermined thresholddistance strictly inferior to the maximum detection range.
 20. Theapparatus of claim 19, wherein the maximum detection range is defined asa minimum value among a plurality of possible maximum detection rangesof the millimeter wave radar sensor system.
 21. The apparatus of claim19, wherein the microphone system is configured to generate the audiobeam for the person only if the detected distance of the person relativeto the apparatus is greater or equal to the predetermined thresholddistance.
 22. The apparatus of claim 19, wherein the microphone systemis configured to generate the audio beam for the person only if thedetected distance of the person relative to the apparatus is inferior orequal to the predetermined threshold distance.
 23. The apparatus ofclaim 18, wherein the millimeter wave radar sensor system is configuredto have a predetermined angular coverage around the apparatus, andwherein the millimeter wave radar sensor includes a plurality ofmillimeter wave radar sensor sub-systems each configured to cover anangular sector of the angular coverage so as to jointly cover the entireangular coverage; and wherein each millimeter wave radar sensorsubsystem has a predetermined threshold distance associated thereto,and, for a person detected in the angular sector of a given millimeterwave radar sensor subsystem, the microphone system is configured togenerate the audio beam for the person as the function at least of thedirection and distance of the person relative to the apparatus only ifthe distance of the person relative to the apparatus satisfies apredetermine criterion relative to the threshold distance associated tothe corresponding millimeter wave radar sensor subsystem.
 24. Theapparatus of claim 23, wherein the respective threshold distances of atleast two millimeter wave radar sensor subsystems are different.
 25. Theapparatus of claim 23, wherein the millimeter wave radar sensor systemhas, for each millimeter wave radar sensor subsystem, a respectivemaximum detection range, and wherein each predetermined thresholddistance is chosen strictly inferior to the maximum detection range ofthe corresponding millimeter wave radar sensor subsystem.
 26. Theapparatus of claim 23, wherein: for a first millimeter wave radar sensorsubsystem, the microphone system is configured to generate the audiobeam for a person detected in the angular sector of the first millimeterwave radar sensor subsystem as the function of the direction of theperson relative to the apparatus only if the distance of the personrelative to the apparatus is greater than or equal to the predeterminedthreshold distance associated to the first millimeter wave radar sensorsubsystem; and for a second millimeter wave radar sensor subsystem, themicrophone system is configured to generate the audio beam for theperson detected in the angular sector of the first millimeter wave radarsensor subsystem as a function of the direction of the person relativeto the apparatus only if the distance of the person relative to theapparatus is inferior or equal to the predetermined threshold distanceassociated to the second millimeter wave radar sensor subsystem.
 27. Theapparatus of claim 18, wherein the apparatus is configured to generate astimulus destined to the person and indicative of the detection of theperson by the apparatus.
 28. A method comprising: using a millimeterwave radar sensor system, detecting a person so as to obtain a directionof the person relative to an apparatus and a distance of the personrelative to the apparatus; using a microphone system configured togenerate one or more audio beam via each of which the microphone systemis configured to receive audio inputs, generating at least one audiobeam as a function at least of said direction and said distance of theperson relative to the apparatus so as to enhance detection of audioinputs generated by the detected person and received by the microphonesystem via the generated audio beam; and receiving audio inputsgenerated by the person via the generated audio beam.
 29. The method ofclaim 28, wherein the millimeter wave radar sensor system has a maximumdetection range, and wherein the audio beam is generated for the personas the function of at least the direction and the distance of the personrelative to the apparatus only if the detected distance of the personrelative to the apparatus is inferior or equal to a predeterminedthreshold distance strictly inferior to the maximum detection range.